1. Field of the Invention
The present invention is related to computer networks and, more particularly, to mining data from network packets.
2. Description of the Prior Art
Current network data mining systems are under increasing pressure to handle higher and higher data throughputs of, e.g., 20 gigabits per second or more, while requiring extraction of data deep within the highest layers of the packet stream. The ever increasing number of protocols and their increased complexity limit the use of hardware data miners. The speed limitations of single-threaded software data miners restricts the maximum data throughput to well under 1 gigabits per second. Hence there is a need to increase the data throughput of software data miners to 20 gigabits per second and beyond.
One tool for analyzing and decoding network traffic is an open-source application known as Wireshark™, which is representative of a range of open-source and commercial products. However, such applications suffer from two major problems. The first problem is that they cannot decode packets at full wire speeds on a continuous basis. The second problem is that they do not have a mechanism for extraction of data from continuous packet streams at full wire speeds.
Prior art patented technology, such as U.S. Pat. No. 7,120,790, addresses the problem of processing packets from high speed networks in two ways. A first method is to reduce the number of packets inspected by applying filters to each packet. A second method is to classify, or summarize, each packet into a much simpler form by limiting the data fields inspected in each packet. The disadvantage to this approach is that analysis is based only on statistical models and not on actual data.
The present invention described herein is capable of decoding and parsing packets at full wire speeds up to 20 gigabits per second while simultaneously extracting targeted data from the packets. The extracted data is made available to the user in a relational database.